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Abstract. We propose an extended forward-backward algorithm for approximating a zero 
of a maximal monotone operator which can be split as the extended sum of two maximal 
monotone operators. We establish the weak convergence in average of the sequence generated 
by the algorithm under very light assumptions on the data. This includes as a special case 
the problem of approximating a minimum point of a proper lower semicontinuous convex 
function on a nonempty closed convex set without any qualification condition. 
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1 Introduction 

Let T~L be a real Hilbert space and let T : % ^ % be a maximal monotone operator. A 
fundamental problem is that of designing efficient algorithms for constructing a zero of T, 
i.e. a solution of the inclusion 



Proximal point algorithms have been introduced by Martinet [8j |9] to solve (pQ) in specific 
situations such as when T is the subdifferential df of a proper convex lower semicontinuous 
function /: in this case the inclusion ([I]) is just the minimization of / over T~L. The general 
situation of an arbitrary maximal monotone operator T was then considered by Rockafellar 
[To] and Brezis-Lions [TJ. See, e.g., Peypouquet-Sorin [13] for a survey on the topic. 

When the maximal monotone operator T admits a decomposition of the form T = A + B 
with A and B maximal monotone on Ti, it is more efficient to use splitting methods which 
combine a step for A and a step for B: see, e.g., Combettes [I] for a unified presentation of 
these methods. A prominent example is the forward-backward algorithm considered by Passty 
|12j . which consists of a forward step on B and a backward (proximal) step on A. A special 
case of this method is the projected subgradient algorithm aimed at solving constrained 
minimization problems. 

There are important examples, however, where the problem to be solved involves two 
maximal monotone operators A and B whose sum A + B is not maximal monotone so that 
the usual splitting methods do not apply. A simple instance of such a situation is the problem 



of minimizing a proper lower semicontinuous convex function / : % — > ] — oo,+oo] over a 
nonempty closed convex subset C of H. This problem can be rewritten as 



£ Tx. 







(3) 



where 5c is the indicator of the set C. When a qualification condition is satisfied, e.g. 
G cor (dom/ — C), then d(f + 5c) = df + d5c, so ([3]) is equivalent to 

Find xen such that G (9/ + <95 c )0), (4) 

where 9/ + 95c is maximal monotone: in this case, standard splitting methods apply. But 
without a qualification condition, such a decomposition of d(f + 5c) as the sum df + d5c 
is not admissible. However, it is always true that d(f + 5c) can be split as the extended 
sum df + e d5c of the two maximal monotone operators df and 85c- Hence, ([3]) is always 
equivalent to 

Find x £ X such that G (df + c d5 c )(x), (5) 

where df + e cWc is maximal monotone. 

The aim of this paper is to present a modified forward-backward splitting algorithm to 
solve the inclusion JT]) for a maximal monotone operator T which can be written as the 
extended sum of two maximal monotone operators. This includes ([2]) as a special case. We 
establish the weak convergence in average of the sequence generated by the algorithm to a 
zero of T under similar hypotheses on the stepsizes as in the classical splitting algorithms in 
Bruck [2!, Lions [7] or Passty [12] . 

Previous extensions of splitting algorithms to treat the case of the extended sum of two 
maximal monotone operators rely heavily on the assumption that the e-enlargement of one 
of the operators is bounded on the solution set as in Moudafi-Thera [11]. Such a strong 
assumption is avoided here. 

2 Tools 

This section presents the main concepts and results we need in the sequel. Most of these tools 
are valid in arbitrary Banach spaces, but we restrict ourselves to the Hilbert space setting 
which suffices for our purpose. 

Let T~L be a real Hilbert space with inner product (., .). A set-valued operator T : H % 
is identified with its graph T C % x T~L, its domain is the set D(T) = {x £ X : Tx ^ 0}. A 
set-valued operator T : % =4 H, or a subset T C H x H, is said to be monotone provided 

(u - v,y - x) > 0, V(x, u), (y, v) G T, 

and maximal monotone provided it is monotone and maximal (under set inclusion) in the 
family of all monotone sets contained in % x7i. 

Let /:%—>■ ]— oo,+oo] be a convex lower semicontinuous function which is proper, 
that is, its effective domain dom/ = {x G % : f(x) < +00} is nonempty. For e > 0, the 
e-subdifferential d £ f of / at x G H is the set: 

d £ f{x) :={ueU: f{x) + (u, y - x) < f(y) + e, Vy G %}. 

The case e = gives the subdifferential of / at x, denoted by df(x). For every e > and 
x G dom/, the set d £ f(x) is nonempty. On the contrary, the set df(x) may be empty at 
some points x G dom/. The subdifferential df is a basic example of a maximal monotone 
operator on % (see Moreau [TO]). 
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The usual (Minkowski) sum df + dg of the subdifferentials of two proper convex lower 
semicontinuous functions /, g : T~L — > ]— oo, +00] is a monotone operator which is not maximal 
monotone in general. Said differently, df + dg is a monotone subset of d(f + g) and the 
inclusion is proper in general without additional assumptions. However, an exact formula for 
the subdifferential of the sum can be given in terms of the sum of the e-subdifferentials of 
each function, namely 

d(f + g)(x) = f]d £ f(x) + d £ g(x), Vxen. (6) 

£>0 

This formula is due to Hiriart-Urruty and Phelps [6] . It shows that the operator on the right 
end side is maximal monotone. 

Let now T : % ^ % be an arbitrary monotone operator. Given e > 0, the e-enlargement 
of T is the operator T £ : H defined as follows: 

T e x :={uen:(v-u,y-x}> -e, V(y, v) G T}. 

For a proper convex lower semicontinuous function /, one has d e f(x) C (df) e (x). For more 
details on this concept, see, e.g., (3J 02 EEH US] and the references therein. The following 
inequality will be useful in the sequel (see [3]): 

Proposition 2.1 Let T : % z4 % be maximal monotone and let E\ > ; 82 > 0. Then 

(v-u,y-x)> -WH + V^) 2 , V(x, u) G T £l , (y, u) G T £2 

As for the subdifferentials, the usual sum (A + B)(x) = Ax + Bx, x G 7i, of two maximal 
monotone operators A, S defines a monotone operator which is not maximal monotone in 
general. Thus, motivated by the above formula ([6]), a concept of extended sum, based on the 
notion of enlargement of a monotone operator, was proposed in [144 115j. namely: 

(A + e B)(x) := p| A £ x + B £ x, Vx G %. (7) 

£>0 

The following result asserts that the extended sum A + e B is indeed a monotone extension 
of the usual sum A + B (see [5]): 

Proposition 2.2 If A, B : Ti 7i are two maximal monotone operators, the extended sum 
A + c B is a monotone operator containing the usual sum A + B. 

It follows from Proposition 12.21 and Formula ([6]) that for any proper lower semicontinuous 
convex functions f,g:H^-] — oo, +00] we have 

d(f + g) = df+ c dg. (8) 

This formula is due to Revalski and Thera [H] (see also [5]). It shows that the extended sum 
df +c dg is a maximal monotone operator. 
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3 Extended forward- backward algorithm 



In what follows, % is a real Hilbert space with inner product (., .) and unit ball B^, and 
T : T~L ^4 T~L is a maximal monotone operator which can be split as the extended sum of two 
maximal monotone operators A and B: 

Tx := f]A £ x + B^x, for all x G U. (9) 

Our aim is to take advantage of this splitting to design a forward-backward algorithm for 
constructing a zero of T, i.e. a solution x G % of the inclusion ([I]). The set of solutions of ([1]) 
is supposed to be nonempty and is denoted by S. 

Let (X n ) be a sequence of positive real numbers defining the stepsizes of a discretization 
scheme, and let a n := Y^k=i denote the step length at stage n. As usual, a n is supposed 
to grow to +oo as n — > oo. Given a sequence (x n ) in H together with a sequence of stepsizes 
(A n ), we define the sequence (x n ) of weighted averages by: 



1 n 

n= — y^XkXk- (10) 



^ fc =i 

We study the extended forward-backward algorithm given by the iteration: 

[ x n+ i = (J + A n ,4) x (x n - X n u n ) with u n G B e "(x n ), Vn G N. 
Our main result is as follows: 

Theorem 3.1 Assume D(^4) C D e> oD(B £ ) and let (x n ) be the sequence generated by the 
iteration [77]) with (A n ) and (e n ) any sequences of positive real numbers verifying 

A„ = oo, ^ A n /3 < oo, e n = A n /3 . 

Assume further that: 

(HI) T/ie sequence (e n u n ) is bounded, 

(H2) For every x G S there is M > suc/i £/ia£ G f| n A £ «x + B £ "x n (M/e n )B H . 

Then, the sequence (x n ) of weighted averages given by ( fiO|) weakly converges to a point in S. 

Proof. The argument relies on Passty's variant of Opial's lemma (see \12\ I13j): 

Opial-Passty's Lemma Let (A n ) be a sequence of positive reals such that ^ A n = oo, (x n ) 
a sequence in % and (x n ) its associated sequence of weighted averages given by M0\) . Let 
S C H be a nonempty closed subset of a Hilbert space H . Assume 

(i) Every weak sequential cluster point of (x n ) belongs to S, 

(ii) lim \ \x n — x\\ exists for every x G S. 
Then, the sequence (x n ) converges to a point in S. 
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The theorem is proved by showing that (i) and (ii) hold. 

(i) Let (x, y) G T, that is, y G A e x + B £ x for every e > 0. Fix e > 0, and take y e = 
?/l,e + 2/2,e with j/i >£ G ^4 £ x, y 2 , £ G and ||y - y e \\ < e. As u n G 5 £ ™x n and y 2 , e G B £ x, 
Proposition 12.11 yields: 

{u n ~ V2,e, x n -x)> -(y/e^ + V^f > -2(e„ + e) (12) 
Since x n - A„u n G x n+ i + A n Ac n+ i by ((IT} and yi )6 G J 4 e x, we get 

X n \ n U n Xn+l 



Vl,£,X n +l - X)>-£, 

An 

equivalently: 

2(x n - x n+1 ,x n+1 -x)> 2X n (u n + y 1>e , x n+1 - x) - 2\ n e. (13) 
By using the equality, valid for all a, b, c G H: 

\\a-cf = ||a-6|| 2 + \\b-c\\ 2 + 2(a-b,b-c), 
we see that (fl~3|) is equivalent to 

\\x n - x\\ 2 - \\x n+ i - x\\ 2 > \\x n - x n+1 \\ 2 + 2\ n (u n + y 1;E ,x n+1 - x) - 2\ n e. (14) 
Now, combining the facts that: 

2\ n (u n + yi, £ ,x n+ i - x n ) > -\\x n+ i - x n \\ 2 - X 2 \\u n + yi, e || 2 , 

and 

{Un + yi,e, X n -x) = (u n - y 2 , £ ,X n - x) + (y £ , X n ~ X) , 

we derive from (fl~4"|) that: 

\\x n ~ X\\ 2 - \\x n +l ~ x\\ 2 > \\x n - X n +l\\ 2 - \\x n +l - X n \\ 2 - \ n \\u n + yi, £ \\ 2 , . 

+2Xn(u n - y2,s,x n - x) + 2X n (y £ ,x n - x) - 2\ n e. 

Then, injecting (|12p into (|15p we obtain 

2X n {y £ ,x n - x) < \\x n - x\\ 2 - \\x n +i - x\\ 2 + \ 2 n \\u n + yi, £ \\ 2 + 6A n e + 4A n e n . (16) 

Summing up these inequalities for n going from 1 to k G N*, and then dividing by = 
En=i A ^ we get: 

2(y £ ,x k - x) < 11X1 -*" 2 + Sti^K+^ll 2 + EtlVgj+j^ 

1/3 

It follows from e n = X n and (HI) that there exists a constant M > such that 

A'hn|| 2 = A^ 3 e 2 ||n n || 2 <A^ 3 M 2 , 
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hence, since ^r/ 3 < oo 



fc 



lim y^\l\\u n + y u \\ 2 <2 lim V A^ 3 M 2 + V A 2 \\ Vl J 2 < +00. 



fc— >oo ^— -" ' fc— >oo 

n=l \n=l n=l 

Finally, since cr^ — > +00, we conclude that 

A 2 llu -l- 112 

fc— >oo U 



lim Sn = i^K + !/i, e „ = _ 



On the other hand, since e n \ 0, there is iV G N such that e n < e for n > N, hence, for k 
large enough: 

Eti A n (6 g + 4 £n ) ^ Eli A w (6 £ + 4 £ra ) + iQ£j 
from which we derive that 

hm sup < 10s. 

fc— >oo "'fc 

Consequently, if x is any weak sequential cluster point of the sequence (x n ), letting k — > 00 
on both sides of (JTTj) yields: 

(y £ ,x-x)<5e, (18) 

and then letting e — > gives: 

(y,x-x)>0. (19) 

Thus, every weak sequential cluster point x of (x n ) satisfies (fl"9|) for every (re, y) G T. The 
maximal monotonicity of T then implies that G Tx, that is, x G 5. 

(ii) Let x G 5. It follows from (H2) that there exist M > and sequences y\ n G A £n x 
and y2,n £ B £n x with e n ||2/2,n|| < M and \\yx >n + U2,n\\ < e n /(l + \\x n - x\\). Without loss of 
generality, we may also assume that e n ||yi,n|| < M and, by (HI), that £ n ||^n|| < M for all n. 

Proceeding as above with y = 0, e = e n and y £n = y± tn + y2,n, we arrive at (fTB"l) . hence 

2X n (y £n ,x n - x) < \\x n - x|| 2 - ||x n+ i - x|| 2 + 2A 2 (||u n || 2 + ||yi, n || 2 ) + 10A n e n . (20) 

1/3 

Then, using £ n — An and taking the above upper bounds into account, we get: 

2A n (y En ,x n -x) < ||x n -x|| 2 - ||x n+ i-x|| 2 + 4A^ 3 M 2 + 10A^ 3 , (21) 
and, since \\y £n \\ < e n /(l + \\x n - x||), 

-2A^ /3 = -2A n e n < -2A n || 2 / £n ||||x n -x|| < 2\ n (y £nl x n - x), (22) 
so, putting (|2T|) and (f22|) together, 

\\x n+ i - xf < \\x n - x|| 2 + A^ /3 (4M 2 + 12). (23) 
Since Yl ^r/ 3 < 00, we derive from (|23l) that lim ||x n — x|| 2 exists hence also lim \\x n — x\\. ■ 
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Remark 3.2 Assumption (HI) is of course satisfied if, as in [12, Theorem 2], the sequence 
(u n ) is supposed to be bounded. Simple examples show that (HI) may hold while (u n ) is not 
bounded, see Example 13.51 below. 

Remark 3.3 Assumption (H2) is satisfied whenever S = (A+B)~ 1 (0), a fortiori when A+B 
is maximal monotone. Indeed, let x £ S. Then, we can find v G Ax n (—Bx), so for every 
small e n > 0, 6 Ax + Bx n (||u||/e n )B^. Hence (H2) holds. However, examples exist with 
(H2) satisfied whereas S ^ (A + £>) _1 (0), see Example 13.51 below. 

Remark 3.4 Assumption (H2) is satisfied if, as in Theorem 2], for every x G S there 
exists e > such that j4 £ x is bounded. Indeed, let x £ S. Since 



there is a sequence (yi >n , U2,n) £ ^4 £n £ x f? £n x such that yi, n +2/2,n - ^ 0- From the boundedness 
of A e x and ^4 e "x C ^4 £ x for large n, we derive that (yi, n ) is bounded, hence we may assume 
that (yi,n) weakly converges to some point y and (y2,n) weakly converges to —y. Since the 
sets A £n x and £> £n x are weakly closed, we derive that y 6 j4 £n x and — y £ -B £n x for every 
n. From the maximal monotonicity of A and B, we conclude that y £ Ax n (—Bx), that is, 
x G (j4 + J B)" 1 (0). Thus, 5 = (A + £) _1 (0)> hence, by Remark E31 (H2) is satisfied. 

Example 3.5 In U = M, let / : R -> R be given by /(a?) = -y/ji for x > 0, /(x) = +oo 
otherwise, and let C = {0}. We consider the problem (|2J) of minimizing the proper lower 
semicontinuous convex function / over the nonempty closed convex set C. As already seen 
in the introduction, this problem amounts to the inclusion ((TJ) where T = d5c + e df is 
maximal monotone by (jHJ). Clearly, the set of solutions of this problem is the nonempty 
set S = {0}. Putting A = 85c and B = df, the algorithm ijTTjl becomes the projected 
approximate subgradient algorithm, noting that (J + 85c) ~ l is the projection Pq onto C: 



We show that the assumptions of Theorem 13.11 are satisfied. 

First, note that D(A) = {0} and D(8 £ f) = dom/ = [0,+oo[, hence D{A) C f] e>Q D{B £ ). 

Next, we have to check the assumptions (HI) and (H2). Easy computations show that 
8 £ 5 C {0) = ]-oo, +oo[ and 8J(0) = ]-oo, -l/4e]. 

For (HI), observe that x n = for every n = 1,2,..., since C = {0}. We may therefore 
choose u n = —l/4e n £ 8 £n f(0) so that e n u n = —1/4. Assumption (HI) is satisfied. 

For (H2), observe that the only point in S is x = 0. Since d s f(0) D [— l/4e, l/4e] = 
{ — l/4e}, we have 



for every e > 0. Assumption (H2) is satisfied. 

Thus, we may apply Theorem 13.11 to derive the convergence in average of the sequence 
generated by the algorithm (I24D to the point minimizing / over C. We point out that no 
sequence u n £ 8 £n f(x n ) is bounded, the operator 85c + df is empty, a fortiori the set 
(85c + 9/)~ 1 (0) is also empty, the sets d e 6c(0) and d e f(0) are not bounded. So none of the 
previous algorithms in [21 [TJ [HI E] is applicable. 




n 



xo £ dom/ 

x n+ i = Pc(x n - X n u n ) with u n £ (8f) £n (x n ), Vn £ N. 



(24) 



= l/4e - l/4e £ 8 £ 5 C {0) + 8 £ f(0) n (l/4e)B K 
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